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"d^" 

^— ^ I Abstract. In this paper we study the problem of description of conjugacy classes in 

^SJ ■ the group SL{n,'L). We expand Gauss Reduction Theory that gives the answer for the 

case n = 2 to the multidimensional case. Reduced Hcssenbcrg matrices now play the 

^ ■ role of reduced matrices. For the case of three-dimensional matrices having a real and 

^0 \ two complex-conjugate eigenvalues we show that perfect Hessenberg matrices distinguish 

*vj ' conjugacy classes asymptotically. An important tool used in our approach is to determine 

^SJ \ minima of Markoff-Davenport characteristics at the vertices of Klein- Voronoi continued 

fractions. We conclude the paper with several open questions arising here. 
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1. Introduction 

In this paper we introduce and investigate a natural structure of the set of conjugacy 
classes in SL{n,Z). Gauss Reduction Theory provides a solution for the simplest case 
n = 2 (see, for instance, in [20], and [14]). We write down a multidimensional analog of 
Gauss reduction an focus on the following question: how good the set of reduced operators 
"approximate" the set of conjugacy classes'! We study this question in the simplest case 
of 5*^(3, Z)-operators with irreducible characteristic polynomial over rational numbers 
having one real and two complex roots. In particularly we obtain that perfect Hessenberg 
matrices distinguish conjugacy classes asymptotically (Theorem 5.11 below). 

According to Gauss Reduction Theory any conjugacy class of SL{2, Z)-operators with 
two real eigenvalues defines a special periodic ordinary continued fraction whose period up 
to cyclic permutations is a complete invariant of the conjugacy class (see in Subsection 5.1). 
Moreover Gauss reduction algorithm applied to any operator of the conjugacy class gives 
a certain reduced operator. The number of reduced operators in a conjugacy class equals 
to the length of the period of the mentioned above ordinary continued fraction. 

We expand to the multidimensional case (n > 3) the idea to use continued fractions 
and their periods as a main tool to construct complete invariants. We put together 
old constrictions of Klein's polyhedron (introduced in [15], [16]) and Voronoi's continued 
fraction (described in [23]) and the ideas of J. A. Buchmann (see in [3]) to obtain a general 
definition of geometric Klein- Voronoi multidimensional continued fraction. In our study 
of SL[3, Z) conjugacy classes we work with periods of Klein- Voronoi continued fractions. 
We use upper Hessenberg matrices as a multidimensional analog of reduced operators. 

Notice that the structure of periods of multidimensional Klein- Voronoi continued frac- 
tions is not clearly known as it is in the classical case. The first attempts to study the 
periods were made by V. I. Arnold [1], E. I. Korkina [18], and author [11] and [12]. 
Nevertheless several characteristics of such periods can be taken as useful invariants. In 
particular we take the values of Markoff-Davenport characteristic on the vertices of Klein- 
Voronoi continued fractions to distinguish SL{3, Z) operators. 

Integer notation. A point (vector) is said to be integer if all its coordinates are 
integers. A segment is said to be integer if its endpoints are integer. An integer length of 
an integer segment is the number of integer points contained in the interior of the segment 
plus one. An m-dimensional plane is said to be integer if the integer vectors contained 
in the plane generate the Abelian group of rank m. A linear transformation is said to 
be integer if it preserves lattice of integer points. Two matrices are said to be integer 
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conjugate if they are conjugate and the transformation matrix corresponds to an integer 
hnear transformation. 

Hessenberg matrices and Hessenberg complexity. A matrix M in the group 
SL{n, M) of the form 



^2,1 ^2,2 
03,2 







V 







0'l,n-2 
<^2,n-2 
0'3,n-2 



Ol,n-l 
0'2,n-l 
0-3,11-1 



0-1,11 \ 

02,n 

03,n 



On—l,n—2 On—ln—1 On—In 







On,n—l 



ttr 



J 



is called an (upper) Hessenberg matrix (such matrices were first studied by K. Hessenberg 
in [10]). The integer 

n-l 



n 



^i+i,i 



\-n—J 



is called the Hessenberg complexity of the matrix M and denoted by ^(M). 

Further, we use the following notation. Let M = (flj,,) be a Hessenberg matrix with 



positive Hessenberg complexity. We say that the matrix M is of Hessenberg type 



{Ol,l, Oi^2\02,l, 02,2) 02,3\ 



-1,1, On-1,2) ■ ■ ■ , On-l,n)- 



Definition 1.1. An integer Hessenberg matrix in SL{n, Z) with irreducible characteristic 
polynomial is said to be perfect ii for any couple of integers {i,j) satisfying 1 < i < j+1 < 
n the following inequalities hold: < aj j < Oj+ij. 

In other words all elements of all the columns except the last column of a perfect 
Hessenberg matrix are nonnegative integers, the maximal elements in these columns are 
the lowest nonzero ones (i.e., flj+ij, j = 1, . . . ,n— 1). 

Note that the set of values of Hessenberg complexity for perfect Hessenberg matrices 
of SL{n, Z) for n > 2 coincides with the set of positive integers. An integer Hessen- 
berg matrix has the unit complexity iff it 02,1 = ■■■ = an,n-i = 1, such matrices are 
called Frobenius matrices. The elements of the last column of a Frobenius matrix are the 
coefficients of the characteristic polynomial multiplied alternatively by ±1. 

Reduced Hessenberg matrices. In this paper we mostly study matrices with irre- 
ducible characteristic polynomials over rational numbers. Any such matrix is conjugate 
to a perfect Hessenberg matrix with positive Hessenberg complexity (see Theorem 2.1 
below). Actually, there are infinitely many perfect Hessenberg matrices integer conjugate 
to a given one. 

Definition 1.2. A perfect Hessenberg matrix W is said to be reduced if for any perfect 
Hessenberg matrix M integer conjugate to W we have 

<,{W) < ^(M). 
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Otherwise we say that the matrix is nonreduced. 

Note that it is possible to have several (but always finitely many) reduced perfect 
Hessenberg matrices conjugate to each other, see Example 5.22. In this case such matrices 
are of the same Hessenberg complexity. This happens also for matrices in SL{2, Z). 

Families of SL{3, Z)-operators with irreducible characteristic polynomials 
and two complex conjugate eigenvalues. We call a matrix with all real eigenval- 
ues (a real and two complex conjugate eigenvalues) a real spectra matrix {non-real spectra 
matrix) or an RS-matrix (NRS-matrix) , for short. 

It is easy to show that any NRS-matrix is conjugate to infinitely many Hessenberg 
matrices. Nevertheless, experiments show that for any fixed Hessenberg type there exist 
only finitely many NRS-matrices that are not perfect. The similar statement does not hold 
for RS-matrices. These allow us to conclude that the results of the experiments are really 
unexpected. 

So, let us focus on families of 5*^(3, Z)-matrices with the same Hessenberg type. From 
Corollary 2.7 it follows that any such family forms a two-dimensional lattice in a two- 
dimensional affine plane in the nine-dimensional space of all matrices. Further in The- 
orem 5.7 we state that the set of NRS-matrices in such plane coincides with the set of 
lattice points of the union of two convex hulls of parabolas except for some points in the 
neighborhood of parabolas (as on Figures 1, 3-6). 

Hence, the set of NRS-matrices for a given Hessenberg type has two asymptotic direc- 
tions corresponding to the parabolas. In Theorem 5.11 we prove that for the sequence of 
SL{'i^'L) -matrices of any ray with an asymptotic direction we have only reduced matrices 
starting from certain moment. 

Description of the paper. In the next section we study basic properties of Hessen- 
berg matrices of arbitrary dimensions. In particular, we discuss existence and finiteness 
of reduced Hessenberg operators integer conjugate to a given one. Further we investi- 
gate families of Hessenberg operators with given Hessenberg type. In Section 3 we study 
the Markoff-Davenport characteristics of Dirichlet orbits (i.e. orbits for the action of the 
Dirichlet group, see also in [2]) and their relations with the Hessenberg complexity. Fur- 
ther in Section 4 we define multidimensional Klein- Voronoi continued fractions, that are 
considered to be a multidimensional analog of ordinary continued fractions in Gauss Re- 
duction Theory. Section 5 contains main results for the case of NRS-matrices in S'L(3, Z). 
Finally, in Section 6 we formulate several questions for further study related to RS- and 
NRS-matrices in 5*^(3, Z) and say a few words about four dimensional case. 

Acknowledgment. The author is grateful to V. I. Arnold for constant attention to this 
work, to H. W. Lenstra and E. I. Pavlovskaya for useful remarks, and University of Leiden 
and Technical University of Graz for the hospitality and excellent working conditions. 

2. Basic properties of Hessenberg matrices 

In this section we study some basic properties of Hessenberg matrices. First, we show 
in Subsections 2.1 and 2.2 that for any ^//(n, Z)-matrix with irreducible characteristic 
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polynomial there exists at least one reduced Hessenberg matrix integer conjugate to the 
given one. In the majority of the observed examples such matrix is unique. Nevertheless, 
in some cases that is not true. In Subsections 2.2 we prove that a Hessenberg matrix is 
uniquely defined by its characteristic polynomial and Hessenberg type. This implies that 
the number of reduced Hessenberg matrices conjugate to a given one is always finite. In 
Subsection 2.3 we describe the lattice structure of the family of Hessenberg matrices with 
a given Hessenberg type. 

2.1. Reduction to Hessenberg matrices. 

Theorem 2.1. Any matrix in SL{n^'L) with irreducible characteristic polynomial is in- 
teger conjugate to a reduced Hessenberg matrix with positive Hessenberg complexity. 

We start with the following lemma. 

Lemma 2.2. Any matrix in SL{n, Z) with irreducible characteristic polynomial is integer 
conjugate to a Hessenberg matrix with positive Hessenberg complexity. 

Proof. Let M be a matrix in SL{n, Z) with irreducible characteristic polynomial, and A 
be a linear operator with matrix M in some fixed integer basis. Take any integer vector 
V of unit integer length and consider a set of vector spaces 

V, = Spa.n{v,A{v),A\v),...,A'-\v)), 

for i = 1, . . . , n (here we denote the span of vectors vi, . . . ,Vm by Span(f i, . . . , Vm))- 
Since for any integer j the vector A^{v) is integer, the spaces Vi are integer. Since the 
characteristic polynomial of A is irreducible, the dimension of Vi equals i and the set of 
all spaces Vi forms a complete fiag in M". 

Let us describe the following basis of M", we denote it by {gi}. We choose v as gi. For 
any i > 1 we chose an integer vector gi & Vi such that gi together with all vectors of Vj_i 
generate an integer lattice in Vi. Notice that the choice of gi is not unique. The vectors 
gi, . . . ,gn form a basis of M" = V„. Let M = {(lij) be the matrix of the operator A in the 
basis {gi}. By construction, this matrix is Hessenberg. 

Since the basis vectors gi, . . . ,gn generate the integer lattice, the matrix M is integer 
conjugate to the matrix M. 

Since the characteristic polynomial of A is irreducible, the integer spaces Vi are not 
invariant subspaces of A. Hence, the integers Oj+i^j are non-zero for i = 1, . . . ,n—l. 
Therefore, the Hessenberg complexity of M is positive. This concludes the proof of Propo- 
sition 2.2. D 

Proof of Theorem 2.1. By Lemma 2.2 it is enough to prove the theorem for a Hessenberg 
matrix with positive Hessenberg complexity. Let M = {aij) in SL{n, Z) be such matrix. 
Suppose also, that A is an operator with matrix M in some integer basis {cj}. Now we 
construct inductively a reduced Hessenberg matrix that is conjugate to M. 

We put ei to be equal to ei. 

Choose Ci^i and ai^i such that 

A{ei) = |a2,i|(sign(a2,i)e2 + Ci^iCi) + ai,iei and < ai,i < |a2,il- 
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Let 

62 = sign(a2,i)e2 + ci,iei. 

Suppose, that for some k we have constructed Cj for alii < k and aij for all i < j < k—1. 

Let us construct e^+i and Oj^fc for i = 1, ■ ■ ■ ,k. Choose Ci^k and ai^k for i = 1, . . . ,k such 

that 

/ fe 

^(cfc) = |afe+i,fc| I sign(afc+i,fc)efe+i + y^ Q.fce^ 

\ i=i 

and < cii^k < cik+i,k for i = 1, . . . ,k. We put 

k 

Cfc+i = sign(afc+i_fc)efe+i + ^Q^fcCi 

and calculate the coefficients ai^k from the expression for A{ek) in the system of linearly 
independent vectors ei, . . . , ik+i- 

The matrix M of the operator A in the basis {cj} is of Hessenberg type 



C^l,!) F2,l| 



'^1,25 Ct2,25 F3,2| 



'^l,n— 1) • • • ) '^n— l,n— 1) \(^n,n—l\ 



By the definition, M is a perfect Hessenberg matrix. 

From the construction it follows that the matrices M and M are integer conjugate. So 
there exists at least one perfect Hessenberg matrix integer conjugate to M. Since the 
set of values of Hessenberg complexity is discrete and bounded from below, there exist 
a reduced Hessenberg matrix among the perfect Hessenberg matrices integer conjugate 
to M. D 

Corollary 2.3. Let A be an SL{n, 7j)-operator and v he any integer vector of unit integer 
length. Then there exists a unique integer basis {gi} such that 

1) 9i = v; 

2) gi e Vi where Vi = Span [v, A{v), A^iv), . . . , A'-^{v)) ; 

3) the matrix M of the operator A in the basis {gi} is perfect Hessenberg. 

Proof. The statement follows directly from the proof algorithm of Lemma 2.2 and Theo- 
rem 2.1. n 

2.2. On identification of Hessenberg matrix. 

Proposition 2.4. Any Hessenberg matrix with positive Hessenberg complexity is uniquely 
defined by its Hessenberg type and the characteristic polynomial. D 

Proof. From definition of Hessenberg matrix it is enough to prove the following. The last 
column in the Hessenberg matrix with positive Hessenberg complexity is uniquely defined 
by its Hessenberg type and the coefficients of the characteristic polynomial. 

Suppose, that we know all the coefficients of the characteristic polynomial and the 
Hessenberg type for some Hessenberg matrix M = {aij). Then the elements aij for 
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1 < i < n, 1 < j < n are uniquely defined by the Hessenberg type of M. Let tlie 
cliaracteristic polynomial of M be 

x" + c„_ix""^ H h cix + Co. 

Direct calculations show that for any k the coefficient Ck is a polynomial in aij variables 
that does not depend on ai „, . . . , afc^„. The unique monomial for Ck containing ttk+i^n is 

n-l \ 

\j=k+l J 

Since the Hessenberg complexity of M is nonzero, we have uniquely defined expressions 
for ak+i,n where k = n—1, n—2, . . . , 0. 

So the last column is uniquely defined. This concludes the proof of the proposition. D 

Corollary 2.5. For any SL{n,Z) -matrix M with irreducible characteristic polynomial 
there exist a finitely many reduced Hessenberg matrices integer conjugate to the given one 
with bounded from above Hessenberg complexity. 

Proof. The existence of reduced Hessenberg matrices integer conjugate to M follows from 
Theorem 2.1. Let the Hessenberg complexity of that reduced matrix equal c. Since the 
conjugate matrices have the same characteristic polynomial and by Proposition 2.4, there 
exists at most one Hessenberg matrix of a given Hessenberg type conjugate to M. The 
number of Hessenberg types with Hessenberg complexity equal to c is finite. Thus, there 
is only a finite number of reduced Hessenberg matrices integer conjugate to M. D 

2.3. A family of Hessenberg matrices with given Hessenberg type. We start with 
two important definitions of integer lattice geometry. 

The integer volume of a simplex a with integer vertices is the index of the sublattice 
generated by the edges of a in the lattice of all integer vectors in the plane spanned by a. 

Take an integer vector v and a A;-dimensional plane tt containing the integer sublattice 
of rank k such that v is not in tt. The integer distance from f to vr is the index of the 
sublattice generated by the integer vectors of the set n U {u} in the whole integer lattice 
of the (A;+l)-dimensional plane spanning v and vr. 

Consider a vector space M" with an integer basis {cj}. For a given Hessenberg type 

^ = (c^l,!) ai,2|a2,l, ^2,2, 02,3] ■ ■ ■ |a„_i^i, . . . , an-1,2, O-n-l^n) 

we associate vertices VkiP) = (ofc,!, • • • , Ofc.fc+i, 0, . . . , 0) for /c = 1, . . . , n—1. Let O be the 
origin. Denote by a{VL) the (n— l)-dimensional simplex with vertices O, fi, . . . ,fra-i. 

Proposition 2.6. A Hessenberg matrix M of type Vt with a vector v in the last column 
is in SLin^Ij) iff the following conditions hold {in the integer lattice of the integer basis 

i) the integer volume of cr{Q) equals one; 

ii) the integer distance from the vector v in the basis {cj} to the integer hyperplane 
containing a{Q) equals one. 
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Proof. Let A be an operator with matrix M in the basis {cj}. 

If M is in SL{n, Z), then A preserves all integer volumes and integer distances. Since 
the integer volume of the simplex with vertices 

0,0+ei,. . .,0+e„_i 

equals one, the volume of the image a{Q) equals one. Since the integer distance from 
the point 0+e„ to the plane spanned by the vectors ei, . . . , e„_i equals one, the integer 
distance from the point (Ai, . . . , A„) to the integer hyperplane containing cr^Q) also equals 
one. 

Suppose, that conditions i) and ii) hold. Then, the operator A takes the integer lattice 
(generated by vectors Ci, . . . , e„) to itself bijectively. Therefore, M is in SL{n, Z). D 

Denote by H{Q) the set of all Hessenberg matrices in SL{n, Z) of the Hessenberg type 
Q. For k = 1, . . . , n—1 we denote by Mk{Q) the matrix with zero first n—1 columns and 
the last one equals to the coordinates of the vector 

(Ofc,!, . . . , ak,2, 0'k,k+l, 0, . . . , 0). 

Corollary 2.7. Let Q be a Hessenberg type satisfying condition i) of Proposition 2.6, and 
Mo be inH{n). Then 

n-l 

Ci , . . . , c„ G Z 



if(fi) = <^Mo + ^QM,(fi) 



Proof. The statement follows directly from Proposition 2.6. D 

We conclude this subsection with a particular example. 

Example 2.8. Let us consider matrices of the Hessenberg type (0, 1|1, 0, 2). All matrices 
of that type form a two-parametric family 



1 n+1 

(0,l|l,0,2)^'^ '" ~ 



H^l'';}\.Am.n)= 110 m 



2 2n + 1 ^ 

with integer parameters m and n. The discriminant of the matrix HL\!-^Q^Jm,n) equals 
—44 — 44n — 5Qmn — 32n + 32m + 16m n + 16mn + 16m n — 56n — 8m + 52m . 

The set of matrices with negative discriminant for the given family coincides with the 
union of integer solutions of the following inequalities: 

2m < — n^ — n — 2 and 2n < m^ + m. 

In Figure 1 we show the family of Hessenberg operators of type (0, 1|1,0,2). The 
square in the intersection of the m-th column and the n-th row corresponds to the matrix 
-^(o'll'i 2>("^'''^)- Black squares correspond to the matrices with reducible characteristic 
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x(-i) = o 




x(i) = o 



Figure 1. The family of matrices of Hessenberg type (0, 1|1, 0, 2). 



polynomials. Light gray squares correspond to the RS-matrices. Dark gray squares cor- 
respond to the nonreduced NRS-matrices. Finally, white squares form the set of reduced 
NRS-Hessenberg matrices. 

There are only 12 dark gray squares corresponding to the nonreduced Hessenberg NRS- 
matrices. We conjecture that there is no other nonreduced NRS-matrices in the family of 
matrices H,Q\L\^^Jm,n). 

3. DiRICHLET ORBITS AND MARKOFF-DAVENPORT CHARACTERISTICS 

The study of Markoff-Davenport characteristic is closely related to the theory of minima 
of absolute values of homogeneous forms with integer coefficients in n- variables of degree n. 
One of the first works on minima of such forms was written by A. Markoff [22] for the 
case n = 2 for the forms decomposable into the product of two real linear forms. Further, 
H. Davenport in series of works [4], [5], [6], [7], and [8] made the first steps for the case of 
decomposable forms for n = 3. 

In this section we give the definition of Markoff-Davenport characteristic. In Subsec- 
tion 3.2 we show a relation between values of Markoff-Davenport characteristic and the 
Hessenberg complexities of Hessenberg matrices of an operator. In Subsection 3.3 we show 
that Markoff-Davenport characteristic is an absolute value of some homogeneous form of 
degree equal to the rank of the operator. 

3.1. Dirichlet orbits. Consider any operator A of SL{n,1j) with irreducible character- 
istic polynomial over the field of rational numbers. The group of all SL{n, Z)-operators 
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commuting with A and having only positive real eigenvalues is called the Dirichlet group 
and denoted by S(A). By Dirichlet unity theorem the Dirichlet group S(A) is homomor- 
phic to some free Abelian group. An orbit of the Dirichlet group consisting of integer 
points is said to be a Dirichlet orbit. 

Example 3.1. Consider an operator Aq with matrix 

/ 1 \ 
10 0-4 
10 1 

y 1 4 y 

The Dirichlet group S(Ao) is isomorphic to Z^ and it is generated by the operators A^, 
(E-Ao)^, and A^+Aq. 

3.2. Markoff-Davenport characteristic. Let us start with the definition. 

Definition 3.2. Consider A G SL{n.,'L). The Markoff-Davenport characteristic (or MD- 
characteristic, for short) of A is a functional defined on M" as follows: the value at a 
vector V is the nonoriented integer volume of the pyramid with vertex at the origin and 
base with vertices v, A{v), . . ., A"'~^{v). Denote the MD-characteristic at v by A{A\v). 

Further in Subsection 3.3 we show that MD-characteristic is a functional coinciding 
with the absolute value of some homogeneous form of degree n. 

Remark 3.3. The MD-characteristic naturally defines a functional over the set of Dirichlet 
orbits for the given operator, since the MD-characteristic at any two vectors of the same 
Dirichlet orbit is the same. 

Proposition 3.4. Consider an operator A with Hessenberg matrix M in some integer 
basis {cj}. Then the Hessenberg complexity ^(M) equals the value of MD-characteristic 

A{A\e,). D 

Proof. Suppose, that the Hessenberg type of the matrix M is 

(Ol,!, 0'1,2\0'2,1, 0-2,2, ^2,3! ' ' ' l^n-l,!, . . . , an-1,2, 0'n~l,n/ ■ 

Denote by Vk the plane Span (t>, A{v), A'^{v), . . . , A''"^(v)). 

Let us inductively show that A''{ei) = i Y[i=i aj,j+i ) Cfe+i + "^fc where Vk is in Vk- 

We have A{ei) = 01,262 + 0'i,iei as the base of induction. 

Suppose, that the statement holds for k = m, i.e., A"^{ei) = {YYu=i ^i,i+i) ^m+i + Vm 
and Vm is in Vm- Let us show the statement for m-\-l. Since M is Hessenberg, A{vm) is 
in Vm+i- Therefore, we have 

A™+i(ei) = aUu ai,^+l] em+ij + A{vJ = 

''m+l \ / / "^ \ 

n '^i,J+l 1 Cm+l + ( ^\Vm) + ( 11 ^i^i+l 1 (A(em,+l)— am+l,m+2eTn+2) 
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The second additive in the last expression is in Kn+i- We have proven our statement. 
Therefore, 

n—l 



i=l 



This concludes the proof of the proposition. 



D 



a 



The proposition implies that for any operator A and any integer d, there exist only 
finitely many Dirichlet orbits with MD-characteristic equal to d. 

3.3. Homogeneous forms associated to SL{n, Z)-operators. Let {cj} be an integer 
basis of M". Consider any SL{n, Z)-operator A with irreducible characteristic polynomial. 
Suppose that it has k real eigenvalues ri,...,rfc and 21 complex conjugate eigenvalues 
ci, ci, . . . , Q, Q, where k + 21 = n. Let us now define a new basis of vectors gi, . . . , gk+21 
in the following way. For i = 1, . . . , /c we choose gi to be an eigenvector corresponding 
to the eigenvalue rj. For j = 1, . . . , / we choose gk+2j-i and gk+2j to be the real and the 
imaginary parts of some complex eigenvector corresponding to the eigenvalue Cj. Consider 
the system of coordinates 

0X1X2 . . . XkYiZiY2Z2 . . . YiZi 

corresponding to the basis {gi}- 
A form 

,i=i j=i 

with nonzero a is said to be associated to the operator A. 

Theorem 3.5. Let A be an S L{n, Z) -operator with irreducible characteristic polynomial. 
Then the MD-characteristic of A is an absolute value of a form associated to A for a 
certain nonzero a. 

Proof. Let us consider the formulas of MD-characteristic of A in the eigen-basis of vectors 

9i, ■ ■ ■ , 9k, gk+i+igk+2, gk+i—igk+2, ■ ■ ■ , gk+21-i+i gk+21, gk+2i-i—igk+2i 

in C", where / = \/—\. Let the coordinates in this eigen-basis be {tj}. 
Then for any vector v = (ti, . . . , t„) we have 

Therefore, 

k I 

A{A\{ti,...,tn)) = a Y[tiY[{tk+2,-ltt+2j 

i=l j=l 
Simple calculations show that a ^ 0. D 



a 



k I 

i=i j=i 
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4. Multidimensional continued fractions in the sense of Klein- Voronoi 

Here we describe multidimensional continued fractions in the sense of Klein- Voronoi. 
We use these continued fractions to find the minimal value of the MD-characteristic for a 
given operator on the lattice of integer points except the origin. 

In 1839 C Hermite [9] posed the problem of generalizing ordinary continued fractions to 
the higher-dimensional case. Since then there were many different definitions generalizing 
different properties of ordinary continued fractions. A nice geometrical generalization of 
ordinary continued fraction for operators with all real eigenvalues was made by F. Klein 
in [15] and [16]. We refer to a nice description, properties, and examples of multidi- 
mensional continued fractions in the sense of Klein to the books by V. I. Arnold [1] and 
G. Lachaud [19] and papers by E. I. Korkina [18], M. L. Kontsevich and Yu. M. Suhov [17], 
and the author [11], [12]. Approximately at the same time of the works by F. Klein 
G. Voronoi in his dissertation [23] introduced a nice geometric algorithmic definition for 
all the cases even for operators with couples of complex conjugate eigenvalues. In [3] 
J. A. Buchmann generalized Voronoi's algorithm making it more convenient for compu- 
tation of fundamental units in orders. 

We use ideas of J. A. Buchmann to define the multidimensional continued fraction in the 
sense of Klein- Voronoi for all the cases. Note that if all the eigenvalues of an operator are 
real numbers then the Klein's multidimensional continued fraction is a continued fractions 
in the sense of Klein- Voronoi. 



4.1. General definitions. Consider an operator A in SL{n,M.) without multiple eigen- 
values. Suppose that it has k real eigenvalues ri, . . . ,rk and 2/ complex conjugate eigen- 
values Ci, ci, . . . , Q, Q, with k + 21 = n. 

Denote by T^A) the set of all real operators commuting with A such that their real 
eigenvalues are all unit and the absolute values for all complex eigenvalues equal one. 
Actually, T'(A) is an abelian group with operation of matrix multiplication. 

For a vector v in R" we denote by Ta{v) the orbit of v with respect of the action of the 
group of operators T^A). If v is in general position with respect to the operator A (i.e. 
it does not lie in invariant planes of A), then Ta{v) is homeomorphic to the /-dimensional 
torus. For a vector of an invariant plane of A the orbit Ta{v) is also homeomorphic to a 
torus of positive dimension not greater than /, or to a point. 

For instance, if f is a real eigenvector, then Ta{v) = {v}. The second example: if v 
is in a real hyperplane spanned by two complex conjugate eigenvectors, then Ta{v) is an 
ellipse. 

As before, let Qi be a real eigenvector with eigenvalue Tj for i = 1, . . . ,k; gk+2j-i and 
gk+2j be vectors corresponding to the real and imaginary parts of some complex eigenvector 
with eigenvalue Cj for j = 1, . . . , /. Again we consider the coordinate system corresponding 
to the basis {gi}: 

0X1X2 . . . XkYiZiY2Z2 . . . YiZi. 
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Denote by vr the (A;+/)-dimensional plane OX1X2 . . . XJ.Y1Y2 ■ ■ -Yi. Let ir^ be the cone 
in the plane vr defined by the equations yi > ior i = 1, ... ,1. For any v the orbit Tj^{v) 
intersects the cone n^ in a unique point. 

Definition 4.1. A point p in the cone 7r+ is said to be n-integerii the orbit Ta(p) contains 
at least one integer point. 

Consider all (real) hyperplanes invariant under the action of the operator A. There are 
exactly k such hyperplanes. In the above coordinates the i-th of them is defined by the 
equation Xi = 0. 

The complement to the union of all invariant hyperplanes in the cone 7r+ consists of 2^ 
arcwise connected components. Consider one of them. 

Definition 4.2. The convex hull of all 7r-integer points except the origin contained in the 
given arcwise connected component is called a factor-sail of the operator A. The set of 
all factor-sails is said to be the factor- continued fraction for the operator A. 
The union of all orbits Tj^{*) in M" represented by the points in the factor-sail is called 
the sail of the operator A. The set of all sails is said to be the continued fraction for the 
operator A (in the sense of Klein- Voronoi). 

The intersection of the factor-sail with a hyperplane in tt is said to be an m-dimensional 
face of the factor-sail if it is homeomorphic to the m-dimensional disc. 

The union of all orbits in M" represented by points in some face of the factor-sail is 
called the orbit-face of the operator A. 

Integer points of the sail are said to be vertices of this sail. 

4.2. Algebraic continued fractions. Consider now an operator A in the group SL{n, Z) 
with irreducible characteristic polynomial. Suppose that it has k real roots ri, . . . , r^ and 
21 complex conjugate roots: Ci,ci, . . . ,ci,ci, where k + 21 = n. In the simplest possible 
cases k+l = 1 any factor-sail of A is a point. If k-\-l > 1, than any factor-sail of A is an 
infinite polyhedral surface homeomorphic to M'^"'"'"^. 

The Dirichlet group S(A) defined in Subsection 3.1 takes any sail of A to itself. By 
Dirichlet unit theorem the group S(A) is homomorphic to Z'^^'^^ and its action on any 
sail is free. The quotient of a sail by the action of S(A) is homeomorphic to the (n— 1)- 
dimensional torus. By a fundamental domain of the sail we mean a collection of open 
orbit-faces such that for any S(A)-orbit of orbit-faces of the sail there exists a unique 
representative in the collection. 

Example 4.3. Let us study an operator A with a Frobenius matrix 









1 


1 





1 





1 


3 



This operator has one real and two complex conjugate eigenvalues. Therefore, the cone 
7r_|_ for A is a two-dimensional half-plane. In Figure 2a the halfplane 7r_|_ is colored in light 
gray and the invariant plane corresponding to the couple of complex eigenvectors is in 
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Figure 2. A tree-dimensional example: a) the cone 7r+ and the eigenplane; 
b) the continued factor-fraction; c) the continued fraction. 

dark gray. The vector shown in Figure 2a with endpoint at the origin is an eigenvector 
of A. 

In Figure 2b we show the cone 7r+. The invariant plane separates 7r+ onto two parts. 
The dots on ir^ are the 7r-integer points. The boundaries of the convex hulls in each part 
of 7r_|_ are two factor-sails. Actually, the sail corresponding to one factor is taken to the 
sail corresponding to the other by the operator —E, where E is an identity operator of 
M3 . 

Finally, in Figure 2c we show one of the sails. Tree orbit-vertices shown in the figure 
corresponds to the vectors (1,0,0), (0,1,0), and (0,0,1). You can see the large dark 
points (0, 1, 0) and (0, 0, 1) lying on the corresponding orbit-vertices. 

The Dirichlet group in our example is homeomorphic to Z and it is generated by A. 
The operator A takes the point (1, 0, 0) and its orbit-vertex to the point (0, 1, 0) and the 
corresponding orbit-vertex. Therefore, a fundamental domain of the operator A contains 
one orbit-vertex and one vertex edge. For instance, we can choose the orbit-vertex corre- 
sponding to the point (1, 0, 0) and the orbit-edge corresponding to the "tube" connecting 
orbit-vectors for the points (1, 0, 0) and (0, 1, 0). 

4.3. Fundamental domains of sails and reduced Hessenberg matrices. Let us 

show how to use Klein- Voronoi continued fractions to find minima of MD-characteristics. 

Theorem 4.4. For any sail of the multidimensional continued fraction in the sense 
of Klein- Voronoi of the operator A we choose one of its fundamental domains. The 
union of all these fundamental domains contains vertices at which the MD- characteristic 
A(A|(Ai, . . . , A„)) attains the minimal values over the integer points of W^ except the 
origin. 

Remark 4.5. Any reduced Hessenberg matrix for the operator A can be constructed start- 
ing from some vertex in a fundamental domain of the multidimensional continued fraction 



,Xk,yi,zi, . 


■,yhZi) =a 


i=l i=l 
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in the sense of Klein- Voronoi. One should construct the Klein- Voronoi continued fraction 
for the operator A. Then one takes all vertices of the chosen fundamental domains for all 
the sails for A, and finds a vertex v with minimal value of the MD-characteristic (note 
that the set of such points is finite). By Corollary 2.3 we reconstruct a basis {cj} where 
Ci = w in which the operator A has a perfect Hessenberg matrix M. By Proposition 3.4 
the Hessenberg complexity of any Hessenberg matrix M' in basis {e^} equals to MD- 
characteristic A(A|e'i). By Theorem 4.4 the vector v is one of the absolute minima of the 
MD-characteristic over the integer points of W^ except the origin. Therefore, the perfect 
Hessenberg matrix M is reduced. 

Proof. Let A be an SL{n, Z)-operator with irreducible characteristic polynomial. By 
Theorem 3.5 there exists a nonzero constant a such that MD-characteristic at any point 
in the system of coordinates OX1X2 . . . XkYiZiY2Z2 . . . YiZi is 

for some positive a. Suppose that the minimal absolute value of F on the set of integer 
points except the origin equals rriQ. 

Choose the coordinates OXi . . . XkYiY2, . . . , Y/ in the cone 7r+. Consider a projection of 
M" to the cone along the Ta{v) orbits. Since we project along the Ta{v) orbits on which 
the MD-characteristic is constant, the projection of the MD-characteristic is well-defined. 
In the chosen coordinates of 7r+ it is written as follows: 

k I 

i=i j=i 

The obtained function is convex in any orthant of the cone 7r+. Let the minimal value of 
the MD-characteristic equals m. Then for any vr-integer point v except the origin the value 
of the function is not less than m. Therefore, the m-th level of this function intersects 
the convex hull of all vr-integer points of some open orthant in its vertices. By definition, 
these vertices are the vertices of some factor-sail, and the corresponding integer points 
are the integer points of the sail. Since the value of the MD-characteristic is invariant 
under the Dirichlet group action, any fundamental domain of the corresponding sail of the 
continued fraction contains vertices at which the MD-characteristic attains the minimal 
value over the integer points of M" except the origin. D 

5. Three-dimensional NRS-matrices 

In this section we study algebraic SL{3, Z)-matrices with irreducible characteristic poly- 
nomials having a real and two complex conjugate roots. We start in Subsection 5.1 with 
formulation of the original statements in two-dimensional case. In Subsection 5.2 we 
describe some properties of the subset of NRS-matrices in the family of all algebraic ma- 
trices with given Hessenberg type. Further in Subsection 5.3 we show that in any ray 
consisting of Hessenberg NRS-matrices only finitely many matrices are not reduced. In 
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Subsection 5.4 we illustrate the obtained results in a few examples. Finally, in Subsec- 
tion 5.5 we give an example of an operator with two distinct reduced perfect Hessenberg 
matrices and show that Hessenberg complexity together with characteristic polynomial 
do not distinguish all the conjugacy classes. 

5.1. A few words about two-dimensional case. First, we recall a situation in two- 
dimensional case. 

5.1.1. Classification of conjugacy classes in S'L(2,Z). The classification of S'L(2, Z)-ope- 
rators with complex spectra is finite. Any such operator is integer conjugate to one of the 
following three operators (that are not conjugate to each other): 

1 1 \ / 1 \ f 1 



-I J \ -I J ' °^ \^ -1 -1 

In the case of operators with multiple eigen- values the complete set of non-conjugate 
representatives is 

''Ik 
1 

Finally let us describe the operators with real spectra without multiple eigenvalues. 



kez 



Definition 5.1. Consider a broken hue . . . Vk-iVkVk+i . . .. The Characteristic sequence 
of this broken line is the following two-side infinite sequence of integers 

. . . , ii{Vk-iVk), HVk^iVkVk+i), ii{VkVk+i), . . . 

where li{PQ) is the integer length of PQ, and Ia{PQR) is the integer angle of PQR (i.e. 
the index of the lattice generated by all integer points of the vectors PQ and QR in the 
whole lattice). 

The Klein- Voronoi continued fraction of an operator A with real spectra is a collection 
of four broken lines (sails). It turns out that all four characteristic sequences for them has 
the same period. 



Theorem 5.2. Consider an S L(2,'Z) -operator H, 



(a,c) 



m] 



i). Let a = 0, b = 1, d = 1. If m > 2 then the operator A has a real spectrum. The 
corresponding period is 

(l,m- 1). 

a). Let b > a > 1, < d < b, m > 1. Let the odd ordinary continued fraction for b/a 
equal 

[ao'-Oi; . . . ; a2k]- 
Then the operator A has a real spectrum. The corresponding period is 

(ao,ai,. ..,a2k,m). 

D 
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For the proofs of this theorem we refer to [13]. 

Notice that for any couple of relatively prime integers (a, b) where 6 > a > there 
exists a couple of integers (c, d), satisfying < d < b and ad~bc = 1. 

For negative values of m in the case a = the periods are (1, |m|— 3). In the case a > 
the periods equal 

(a2fc, . . . ,ai, Oq, Imi — 2), 
where [a'oia'i; . . . ; 02^] — is the odd ordinary continued fraction for b/{b — a). 

Remark 5.3. The described periods form the complete invariant of operators. Here all 
cychcally shifted periods are considered to be the same (e.g. (010203) = (020301) = 
(03O1O2)), while the multiple periods not (e.g. (01O2) 7^ (01O2O1O2)). 

For further information about the two-dimensional case we refer the reader, for instance, 

to the works [20], [21], and [13]. 

5.1.2. Family of m,atrices with fixed Hessenberg type. In the next subsections we generalize 
the following statement. 

Statement 5.4. Consider the fam^ily of matrices H^' {m) with parameter m . Then almost 
all matrices in this family are reduced and has two real eigenvalues. D 

The proof can be easily deduced from Theorem 6 of [13]. 

Example 5.5. Consider the matrices of the Hessenberg type (2,5). Let 

f7(i-3). N_ /^2 l + 2m 

The Hessenberg complexity of all these matrices is 5. There are no matrices with reducible 
characteristic polynomial in the family. The matrix with m = —1 has non-real spectrum, 
and the matrices with m = —3, —2, 0, 1 are nonreduced. All the rest matrices of that type 
are reduced, have two real eigenvalues, and with irreducible characteristic polynomial. 

The corresponding periods for the case of m > 2 are: (2, 1, 1, m); for the case m < —4 
they are (1, 2, 1, — m — 2). 

5.2. Asymptotic structure of the subset of perfect Hessenberg NRS-matrices. 

Note that an 5*^(3, Z)-matrix has a reducible characteristic polynomial iff one of its eigen- 
values equals to ±1. 

In Theorem 2.1 we have shown that any SL{3, Z)-matrix is integer conjugate to one of 
the reduced Hessenberg matrices of the form: 



Ol,l 


Ol,2 


ai,3 


^2,1 


02,2 


02,3 





03,2 


03,3 



where < 01^2 < 02,1, < 01^2 < 03^2, and < 02,2 < 03,2- 

Consider the family H{Q) for a Hessenberg type fl = (oi^i, 02,1(01^2, 02,2, 03,2)- Denote 
the subset of all NRS-matrices in i7(fi) by iVi?S'(f]). 
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Figure 3. The family of matrices of Hessenberg type (0, 1|0, 0, 1). 



For the given Hessenberg type Q we choose integers ai^s, 02,3, and 03,3 such that the 



determinant of the matrix Mq 
set H[Q) is a family 



^«Jy 



is a unit. As we have shown in Corollary 2.7 the 




ai,2 ai,im + 01^2^ + 01,3 
02,2 a2,im + 02, 2« + ^2,3 
03,2 03,2^ + a3_3 



m E 1^,71 E Ij 



By H^{mo, no) we denote the matrix with integer parameters m = rriQ and n = no in the 
family H{Q). Later in the proofs we use also matrices i7Q(mo,no) with real parameters, 
i.e., the matrices in the real afiine two-dimensional plane spanned by H{Q) in the real 
affine space of all matrices. We denote by OMN the coordinate system corresponding to 
the parameters {m,n). The origin O here corresponds to the matrix Hq{0,0). 

Denote the discriminant of the characteristic polynomial of H^{m,n) by VQ{m,n). So 
the set NRS{Q) coincides with the set of integer solutions of the inequality 



Vo(m,n) <0 



in variables m and n. 



Example 5.6. We show in Figure 3 the subset of NRS-matrices NRS{fi) for the Hessen- 
berg type Q = (0, 1|0, 0, 1). For this example we choose (01,3, 02,3, 03,3) = (0, 0, 1). 

In both examples shown in Figure 3 and in Figure 1 on page 9 we see that the set 
NRS{Q) "looks like" the set of integer points in the union of the convex hulls of two 
parabolas. 
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Let US formulate a precise statement. Suppose that the characteristic polynomial of the 
matrix H^'{0,0) = (dij) in the variable t equals 

-t^ + hit^ - h^t + 63. 

In the case of SL{'i^'L) we have 63 = 1, nevertheless we continue to write 63 for possible 
use for matrices with distinct determinants. 

For a family HQ{m,n) we define the following two quadratic functions 

ain^ — Pin — 71; 

'02,1^72 — ai,in\ " 
— 0:2 1 ' 
02,1 



Pi,n[m,n) 
P2,n{rn,n) 



m - 
n 



02,1 



1 



8-2 



02 iiTT- ~ oi in 



where 



tti 

71 



03,2 



4a2,i 

Ol,l — 02,2 ~ ^3,3 



0:2 



02,1 
03,202,1 



72, 



462 



2a2,i 



402,103,2 



P2 = - 



72 



"2 



463 
b2_ 

26, 



46163 



402,103,263 



Denote by B^{0) the interior of the circle of radius R centered at the origin (0, 0) in 
the real plane OMN of the family HQ{m,n). We denote also 

As = {(m,n)|(pi,n(m,n) - e){p2,n{m,n) - e) < 0}. 

Theorem 5.7. For any positive e there exists R > such that in the complement to 
Br{0) the following inclusions hold 

A, c NRS{n) c A_,. 

Before to start the proof we make the following remark. The set NRS{Q) is defined by 
the inequality 

In the left part of the inequality there is a polynomial of degree 4 in variables m and 
n. Note that the product 16a2 io| 263(^1, n(m,n)j92,n(^,'^)) is a good approximation to 
X'^" {m, n) at infinity: the polynomial 

Vn{m,ny° - 1602,103 363 (pi,Q(m,n)p2,f7("^,n)) 

is a polynomial of degree 2 in variables m and n. 

Lemma 5.8. The curve VJ-^L^-^Amjn 



is contained in the domain defined by the 



inequalities: 



(m^ - 4n + 3)(n2 + 4m + 3) > 
(m^ -An- 3)(n2 + 4m - 3) - 72 < 



Remark. Lemma 5.8 implies that the curve ^(o'llo 1 
tubular neighborhood of the curve 

(m^ — 4n)(n^ + 4m) = C 



(m,n) 



is contained in some 



20 OLEG KARPENKOV 

Proof. Note that 

"^(o'llo i)("^' '^) ~ (^^ ^ 4n)(n^ + 4m) — 2mn — 27. 
Thus, we have 

^(Moki)(^'^)-(^^"4n + 3)(n=^ + 4m + 3) = -2(n - 3)^ - 2(m + 3)^ - (n + m)=^ < 0, 
and 
^So'?lo^o 1) (^' ^) - (^^ - 4n - 3)(n2 + 4m - 3) + 72 = 2(n - 3)^ + 2(m + 3)^ + (n - m)^ > 0. 



(0,1|0,0,1)* 

^(O.'lio'o,!)^"^'^'' 



Therefore, the curve P/n'n'nn i\(^^, n) = is contained in the domain defined in the lemma. 

D 



n 



Lemma 5.9. For any Q = {an, a2i\ai2, 022, (132) there exists a {not necessarily integer) 
affine transformation of the plane OMN taking the curve V^[m,n) = to the curve 

^(i'?i!l?o,i)K^) = o- 

Proof. Let H^{0,Oy° = (aij). Note that a matrix H^'{m,n) is rational conjugate to the 
matrix 

-^(0,1 |'o,o,i> 1*^23 '^32 ~ Oil O33 + ai2a2i — a22 0.33 — 0.11 O22 + (i2i032Tn — ail a32n, Oii + ^22 + ^33 + a32n) 

by the matrix 

(1 ai,i aj-^ + 01,202,1 

02,1 0i,i02,i + 02,1^2,2 

02,103,2 

Since both matrices 

i/^«(m,n) and {X'^'y' {Hnim,n))X. 

have the equivalent characteristic polynomials, their discriminants coincide. Therefore, 
the curve V^ = is mapped to the curve PL'-^Lq -^>(m, n) = bijectively. 

In OMN coordinates this map corresponds to the following affine transformation 

^ \ ^ f a2i032m - Oii032n \ , f ^23032 - O11O33 + O12O21 - O22O33 - 011022 
n J y 032n J \ On + O22 + O33 

This completes the proof of the lemma. D 

Proof of Theorem 5.7. Consider a family of matrices i7^°(— pi,n(0, t) + e,t) with real 
parameter t. Direct calculations show that for 5 7^ the discriminant of the matrices for 
this family is a polynomial of the forth degree in variable t, and 

^^"(-Pi,n(0, t) +e,t) = ^02,io|2^t^ + Oit^). 

Therefore, there exists a neighborhood of infinity with respect to the variable t such that 
the function 'DJ^"(— pi,q(0, t) + e,t) is positive for positive e in the neighborhood, and 
negative for negative e. 
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Hence for a given e there exists a sufficiently large Ni = Ni{e) such that for any t > Ni 
there exists a solution of the equation V^{m,n) = at the segment with endpoints 

(-Pi,n(0,t) +e,t) and (-pi,f^(0,t) -£,t) 

of the plane OMN. 

Now we examine the family in variable t for the second parabola: 



/J^» t-ai,ip2,n(t,0)- ,r-^, g,-a2.iP2.f7(t,0) 



By the same reasons, for a given e there exists a sufficiently large N2 = N2{e) such that 
for any t > N2 there exists a solution of the equation ^^[m, n)"" = at the segment with 
endpoints 

(t-ai^iP2,n(t, 0) , "'■' 2 g, -a2,iP2,n(t, 0) ^^M==e) and 

(t-ai,iP2,Q(t, 0)+^==^£, -a2,iP2,n(t, 0)+ /J''], g) 

V"l,l+"2,1 V 1>1"^ 2,1 

of the plane OMN. 

We have shown that for any of the four branches of the two parabolas defined by pi ^ 
and p2,n there exists (at least) one branch of T>Q^{m,n)'"° = contained in the e-tube of 
the chosen parabolic branch if we are far enough from the origin. 

In Lemma 5.8 we have obtained that T),Q\LQ-^^Am,n) = is contained in some tubu- 
lar neighborhood of pi^(o,i|o,o,i>(^;'^)P2,(o,i|o,o,i>(^;''^) — 0- Then by Lemma 5.9 we have 
that the curve V^{m,n) = is contained in some tubular neighborhood of the curve 
Pi,Q.{f^-,n)p2,Q.{f^-,n) = outside some ball centered at the origin. Finally, by Viet Theo- 
rem, the intersection of the curve V^ {771,11) = with each of the parallel lines 

ai 1 -|- 02,1 , 

It '■ n — 7n = t 

02,1 

contains at most four points. Therefore, there exists sufficiently large T such that for 
any t > T the intersection of the curve V^ (m, n) = and it contains exactly four points 
corresponding to the branches of the parabolas Pi,Q(m, n) = and P2,Q{m-,n) = lying 
in A_, \ A,. 

Hence, there exists R = R{e, Ni, N2, T) such that in the complement to the ball Bji{0) 
we have 

A, C NRS{Q) C A_,. 
The proof of Theorem 5.7 is completed. D 

5.3. Asymptotic behaviour of Klein- Voronoi continued fractions for the NRS- 
case. In the previous subsection we have shown that for any Hessenberg type ^2 = 
(ai,i, a2,i|ai,2, ^2,2, 03,2) the set NRS{Q) almost coincides with the union of the convex 
hulls of two parabolas. We say that the set NRS{Q) has "two asymptotic directions" 
that correspond to the symmetry lines of the parabolas. These directions are defined by 
the vectors (—1,0) and (01^1,02,1). To be precise, an integer ray in OMN is said to be 
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an NRS-ray if all its integer points correspond to reduced Hessenberg NRS-matrices. A 
direction is said to be asymptotic for the set NRS{Q) if there exists an NRS-ray with this 
direction. 

Theorem 5.7 implies the following proposition. 

Proposition 5.10. There are exactly two asymptotic directions for the set NRS{Q), they 
are defined by the vectors (—1,0) and (01,1,02,1)- D 

We will use the following notation. Fix a basis {ei, 62, 63} in M^, consider a Hessenberg 
type fi = (ai,i, 02, i|ai,2, 02,2, 03,2) and choose an integer appropriate vector vq. For any 
couple of real numbers {m,n) and nonnegative t we denote the operators Rl^^i'^) and 
i?^^(t) with matrices: 

iJ^" {m — t, n) and iJ^" {m + ai,it, n + 02,1^) 

respectively. Denote by R]fn and R]fn the two corresponding families of operators 
with nonnegative parameter t. The directions of these rays are asymptotic: (—1,0) and 
(ai, 1,02,1) respectively. 
Now we are ready to formulate and to prove the following theorem. 

Theorem 5.11. On asymptotics for NRS-rays. Any NRS-ray for the set NRS{VL) 
contains only finitely many nonreduced matrices. Any such ray contains only finitely many 
reduced matrices integer conjugate to some other reduced matrices. 

Example 5.12. Any NRS-ray for the Hessenberg type (0, 1|0, 0, 1) contains only reduced 
perfect matrices. Experiments show that any NRS-ray for (0,1|1,0,2) contains at most 
one nonreduced matrix. 

Proof. First, we prove the theorem for NRS-rays with asymptotic direction (—1,0). 

Let us begin with the case of real matrices of Hessenberg type Qq = (0, 1|0, 0, 1). Such 
matrices form a family H{Qq) with real parameters m and n as before: 

^(0,110,0,1) ("^'^) = 

Lemma 5.13. Let Rl^^ ^^ ^ family of operators having NRS-matrices with nonnegative 
integer parameter t. Then for any e > there exists C > such that for any t > C the 
convex hull of the union of two orbit-vertices 

Ti.fio.,-, (1,0,0) and T i.no.,-,(0, 1, 0) 

is contained in the e-tubular neighborhood of the convex hull of three points (1,0,0), 
(0,1,0), (0,-1,0). 

Proof. Let us find the asymptotics of eigenvectors and eigenplanes for operators Rl^'pn (t) 
while t tends to -|-oo. Denote any real eigenvector of R]f'°{t) by e(t). We have 

e(t)=/i((l,0,0) + O(ri)) 
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for some nonzero real /i. 

Consider the unique invariant real plane of the operator R^^n (^) (it corresponds to the 
couple of complex conjugate eigenvalues). Note that this plane is a union of all closed 
orbits of Rli^nit). Any such orbit is an ellipse with axes \gtna.x(t) and Xgmmit) for some 
positive real number A, where 

Wx(t) = (0,t,0)+O(l), 

^min(t) = (0,0,tl/2)+O(t-V2). 

Actually, the vectors gmax{t)^I gmm{t) are two complex eigenvectors of Rl^ni^)- For the 
ratio of the lengths of maximal and minimal axes of any orbit we have the following 
asymptotic estimate: 



Since 



A|Wx(t)| ^ 1/2^0(1^1^1/2) 
M9mm{t)\ 

(1,0,0) -ie(t) = 0(|t|-i), 

/i 



the minimal axis of the orbit-vertex T i.oq, ,(1, 0, 0) is asymptotically not greater than 

0{t~^). Therefore, the length of the maximal axis is asymptotically not greater than some 
function 0(|t|~^/^). Hence, the orbit of the point (1,0,0) is contained in the (Ci|t|~^/^)- 
ball of the point (1, 0, 0), where Ci is a constant that does not depend on t. 
We have 

(0,l,0)-^(?^ax(t)=O(|t|-l). 

Therefore, the length of the maximal axis of the orbit- vertex T„i,s2,w . (1, 0, 0) is asymp- 

totically not greater than some 1 + 0(t~^/^). Hence, the length of the minimal axis is 
asymptotically not greater than some 0(|t|~^/^). This implies that the orbit of the point 
(0, 1,0) is contained in the (C2|t|^^/^)-tubular neighborhood of the segment with vertices 
(0, 1, 0) and (0, —1, 0), where C2 is a constant that does not depend on t. 
Therefore, the convex hull of the union of two orbit-vertices 

Tj.i,no,Jl,0,0) and Ti,n„ (0,1,0) 

is contained in the C-tubular neighborhood of the triangle with vertices (1, 0, 0), (0, 1, 0), 
(0, —1, 0), where C = |t|~^/^ max(Ci, C2). This concludes the proof of the lemma. D 

Let us now formulate a similar statement for the general case of Hessenberg operators. 

Corollary 5.14. Let R]^n ^^ ^ family of operators having NRS-matrices with integer 
parameter t > 0. Then for any e > there exists C > such that for any t > C the 
convex hull of the union of two orbit-vertices 

Tfjhn^, (1,0,0) and T i,no,,Jai,i, 02,1, 0) 
is contained in the e-tubular neighborhood of the convex hull of three points (1,0,0), 

(01,1,02,1,0), (-01,1,-02,1,0). 
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Proof. Denote Q = (01,1,02,11^1,2,02,2503^2) and choose 

—03,201,1 Oi,i02,2 ~ 02,lOi,2 

X = I 03,2 — Ol,l — 02,2 

1 




Direct calculation shows that 



XHn{m - t, n)X ^ = -f/'(o^i|o,o,i) ( h{m, n) 



-,hino] 



02,103,2 

where /i and I2 are linear functions with coefficients depending only on Oi,i, 02,1, 01,2, 02,2, 
and 03,2. 

Therefore, the family -R^^ after the described change of coordinates and a homothety 

is taken to the family of matrices Rj^^{t) of the type (0, 1|0, 0, 1) for certain m and n. 

Lemma 5.13 implies the following. For any e > there exists a positive constant such 
that for any t greater than this constant the convex hull of the union of two orbit- vertices 

T„h^o(A^,0,0) and Tui^ (0,1,0) 

is contained in the e-tubular neighborhood of the triangle with vertices (1, 0, 0), (0, 1, 0), 
(0,-1,0). 

Now if we reformulate the last statement for the family of operators in old coordinates, 
then we get the statement of the corollary. D 

In the algebraic case we have the following statement. 

Proposition 5.15. Let {rn,n) be a couple of integers such that the ray -R^^ contains only 
operators having NRS-matrices for integer parameter t > 0. Then there exists C > such 
that for any integer t > C there exists a fundamental domain for a Klein- Voronoi sail of 
the operator Rlf'^it) such that all {integer) orbit-vertices of this fundamental domain are 
contained in the set of all integer orbits corresponding to the integer points in the convex 
hull of three points (1, 0, 0), (oi,i, 02,1, 0), and (—01,1, —02,1, 0). 

We use the following general lemma on continued fractions. 

Take an operator A having an NRS-matrix in SL{'i, Z) and any integer point x distinct 
from the origin. Denote by T\[p) the convex hull of the union of two orbits corresponding 
to the points p and A{p). For any integer k we denote by T\{p) the set A^{V\[x)). Let 



^A{p) = [JT\{p). 



kez 

Lemma 5.16. Consider A G SL{3,'Z) with NRS-matrix and let p be any integer point 
distinct from the origin. Then one of the Klein- Voronoi sails for A is contained in the set 

^Aip). 

Proof. Note that the set Ta{p) is a union of orbits. Let us project Ta{p) to the halfplane 
7r_|_, see in Figure 2 above. The set Ta{p) projects to the closure of the complement of the 
convex hull for the points 7i{A^{p)) for all integer k in the angle defined by eigenspaces. 
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Since all the points A^{p) are integer, their convex hull is contained in the convex hull 
of all points corresponding to integer orbits in the angle. Hence 7r(r^(j9)) contains the 
projection of the sail. Therefore, the set Ta{p) contains one of the sails. D 

Corollary 5.17. Let A be an operator in SL{3,'Z) having an NRS-matrix and p — an 
integer point distinct from the origin. Then there exists a fundamental domain for one of 
the Klein- Voronoi sails for an operator A with all (integer) orbit-vertices contained in the 
setT\{p). 

Proof. Note that T\{p) is a fundamental domain of Ta{p) for the action of the Dirichlet 
group S(A). Hence, Ta{p) contains all orbits of orbit-vertices for the action of S(A). D 

Proof of Proposition 5.15. We note that the operator Rl^^{t) takes the point (1,0,0) 
to the point (ai,i, 02,1, 0). Therefore, the convex hull of the union of two orbit- vertices 

(we denote it by W{t)) coincides with the set T^in (1, 0, 0). 

From Corollary 5.17 it follows that there exists a fundamental domain for a sail with all 
its orbit- vertices contained in W{t). Choose a sufficiently small Eq such that the ^o-tubular 
neighborhood of the triangle with vertices (1, 0, 0), (ai,i, 02,1, 0), and (— ai,i, —02,!) 0) does 
not contain integer points distinct from the points of the triangle. From Corollary 5.14 it 
follows that for a sufficiently large t the set W{t) is contained in the eo-tubular neighbor- 
hood of the triangle. This implies the statement of Proposition 5.15. D 

Now let us study the remaining case of the rays of matrices with asymptotic direction 
(ai,i, 02,1)- We remind that Qq = (0, 1|0, 0, 1). 

Lemma 5.18. Let R^^ ^^ ^'^ NRS-ray. Then for any e > there exists C > such 
that for any t > C the convex hull of the union of two orbit-vertices T 2,n^^, {1, 0, 0) and 

Tp2,s2o^,x(0, 1, 0) is contained in the e-tubular neighborhood of the convex hull of three points 

(1,0,0), (-1,0,0), (0,1,0). 

Proof. First, we note that the continued fractions for the operators A and A^^ coincide. 
Secondly, the following holds: 

^(o,'i°|o,o,i>("^' "- + ^) = -^^(o,'i°|o,o,i>(~^ ~ ^' -"^)-^~^ 
where 

X = 




Thus, in the new coordinates we obtain the equivalent statement for the ray -R_'„'Lm(^)- 
Now Lemma 5.18 follows directly from Lemma 5.13. D 
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Corollary 5.19. Let Q = {ai^i,a2^i\ai^2, (12,2,0,3,2) o.^'^d R^f^rX'^) ^^ a family of operators 
having NRS-matrices with parameter t > 0. Then for any e > there exists C > such 
that for any t > C the convex hull of the union of two orbit-vertices 

is contained in the e-tubular neighborhood of the triangle with vertices (1, 0, 0), (—1, 0, 0), 
and (ai,i, 02,1, 0). D 

Proposition 5.20. Let {m,n) be a couple of integers, such that the family -R^^(t) con- 
tains only operators having NRS-matrices. Then there exists C > such that for any 
t > C there exists a fundamental domain for a Klein- Voronoi sail of the operator R^^{t) 
such that all orbit-vertices of this fundamental domain are contained in the set of all in- 
teger orbits corresponding to the integer points in the convex hull of three points (1,0,0), 
(—1,0,0), anc? (oi^i, a2,i, 0). D 

Remark We omit the proofs of Corollary 5.19 and Proposition 5.20, since they repeat 
the proofs of Corollary 5.14 and Proposition 5.15. 

Now we prove Theorem 5.11. 

Step 1. Let A be an operator with Hessenberg matrix M in 5*^(3, Z). By Propo- 
sition 3.4 the Hessenberg complexity of the Hessenberg matrix M coincides with the 
MD-characteristic A(A|(1, 0, 0)). Therefore, the Hessenberg matrix M is reduced if and 
only if the MD-characteristic of A attains the minimal possible absolute value on the 
integer lattice except the origin exactly at point (1,0,0). 

Step 2. By Theorem 4.4 we know that all minima of the set of absolute values for the 
MD-characteristic of A are attained at integer points of the Klein- Voronoi sails for A. 

Step 3. In the case of a three-dimensional operator A with two complex-conjugate 
eigenvalues, the continued fraction of A contains exactly two sails that are symmetric 
with respect to the transformation —E. Therefore, both sails of A has minima of the 
MD-characteristic on an integer lattice except the origin. This allows us to consider one 
sail. By Theorem 4.4 we can restrict the search of the minimal absolute value of the 
MD-characteristic to one of the fundamental domains of the chosen sail. 

Step 4-1- The case of NRS-rays with asymptotic direction (—1,0). 

By Proposition 5.15 there exists a positive constant such that for any integer t greater 
than this constant all integer points of one of the fundamental domains for R]fn{t) are 
contained in the convex hull of three points (1, 0, 0), (ai^i, 02,1, 0), and (— ai^i, -02,1, 0). 

This triangle contains only finitely many integer points, all of them have the last coor- 
dinate equal to zero. The value of the MD-characteristic for the points of type (x, y, 0) 
equals: 

(02,10; - ai^iy)al2yH + C, 

where the constant C does not depend on t, it depends only on x, y, and the elements of 
the matrix H^{m — t, n) for the operator R]^^{t). 

So for any point (x, y, 0) the MD-characteristic is linear with respect to the parameter t, 
and it increases with growth of t. The only exceptions are the points of type A(1,0,0) 
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and /i(ai,i, 0,2,1, 0) (for integers A and fi). The values of MD-characteristic are constant in 
these points with respect to the parameter t. 

Since there are finitely many integer points in the triangle (1,0,0), (ai^i, 02,1, 0), and 
(—01^1,-02,1,0), for sufficiently large t the MD-characteristic at points of the triangle 
attains the minima at (1, 0, 0) or at (ai,i, 02,1, 0). Since Rj^^it) takes the point (1, 0, 0) to 
the point (ai,i, 02,1, 0), the values of the MD-characteristic at (1, 0, 0) and at (oi^i, 02,1, 0) 
coincide. 

Therefore, for sufficiently large m the matrix 

^^'"^ , Am-t,n) 

is always reduced and there are no other reduced matrices integer congruent to the given 
one. We have proved the statement of Theorem 5.11 for any NRS-ray with asymptotic 
direction (—1, 0). 

Step 4- 2- The case of NRS-ray s with asymptotic direction (oi^i, 02,1)- This case is similar 
to the case of NRS-rays with asymptotic direction (—1,0). We leave the details of the 
proof to the reader. 

Proof of Theorem 5.11 is completed. D 

5.4. Examples of NRS-matrices for a given Hessenberg type. In this subsection 
we bring together some examples of families NRS{Vt) for the Hessenberg types: 

(0,1|0,0,1), (0,1|1,0,2), (0,1|1,1,2), (0,1|1,0,3), and (1,2|1,1,3). 

5.4.1. Hessenberg perfect NRS-matrices H,Q\L\^^Am^n). The Hessenberg complexity of 
all these matrices is one, and, therefore, they are all reduced, see the family on Figure 3 
on page 18. 

5.4.2. Hessenberg perfect NRS-matrices HJ^,^\^^Am^n). The Hessenberg complexity of 
these matrices equals 2. Experiments show that 12 of such matrices are nonreduced, see 
the family in Figure 1 on page 9. It is conjectured that all others Hessenberg matrices of 
A^i?^((0,l|l,0,2)) are reduced. 

In Figures 4, 5, and 6 the dark gray squares corresponds to nonreduced operators. We 
also fill with gray the squares corresponding to reduced Hessenberg matrices that are n-th 
powers (where n > 2) of some integer matrices. 

5.4.3. Hessenberg perfect NRS-matrices H,q\i\^ 2)("^' ^)- ^^^ ^^^ family in Figure 4. The 
Hessenberg complexity of these matrices equals 2. We have found 12 nonreduced matrices 
in the family. It is conjectured that all other Hessenberg matrices of NRS(^{0, 1|1, 1, 2)) 
are reduced. 

5.4.4. Hessenberg perfect NRS-matrices HL'-^L\^^Jm,n). See the family in Figure 5. The 
Hessenberg complexity of these matrices equals 3. We have found 6 nonreduced matrices 
in the family. It is conjectured that all other Hessenberg matrices of NRS(^{0, 1|1,0,3)) 
are reduced. 
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m—n = 2 



x(i) = o 



Figure 4. The family of Hessenberg matrices H,Q\^^\^Am^n) 




m—n = 1 



x(i) = o 



(1,0,3) 



Figure 5. The family of Hessenberg matrices HL^L^^JTri^n). 
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Figure 6. The family of Hessenberg matrices -f^/V2|'i i 3\- 

5.4.5. Hessenberg perfect NRS-matrices Hi'J^-^L(m, n). See the family in Figure 6. This 
is a more complicated example of a family of Hessenberg perfect NRS-matrices, their com- 
plexity equals 12. We have found 27 nonreduced matrices in the family. It is conjectured 
that all other Hessenberg matrices of NRS(^{1, 2|1, 1, 3)) are reduced. 

5.5. Two examples of couples of Hessenberg matrices. Let us give an example of 
an operator with two distinct reduced perfect Hessenberg matrices. 

Example 5.21. The following two matrices 



1 3 \ 




/ 2 5 


1 


and 


1 1 2 


3 8/ 




\ 3 7 



are not integer conjugate but have the same Hessenberg complexity equal to 3 and equiv- 
alent characteristic polynomials. 

The following examples shows that Hessenberg complexity together with characteristic 
polynomial do not distinguish all the conjugacy classes. 

Example 5.22. The matrices Mi M2, where 

Ml = I 1 I and M, 
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are both reduced Hessenberg matrices (with Hessenberg complexity equal to 3) of the 
same operator A in two different integer bases. The reason for that is the following. The 
sail of A in the basis of the matrix Mi contains two different integer vertices pi = (—1, 0, 0) 
and p2 = (0, 1, —1) in a fundamental domain that are not in the same Dirichlet orbit. 

6. Open problems and questions 

In this section we discuss several directions for further study. In Subsection 6.1 we 
make a brief analysis of all the observed in the previous section examples in NRS-case and 
formulate related questions. In Subsection 6.2 we observe the cases of three dimensional 
RS-matrices. Finally we say a few words about four dimensional case in Subsection 6.3. 



6.1. Three dimensional NRS-matrices. As we have shown in Theorem 5.11 the num- 
ber of nonreduced matrices in NRS-rays is always finite. Actually, from the experiments 
we conjecture that a stronger statement is true. 

Problem 1. Let us fix some perfect Hessenberg type of three-dimensional matrices. Is it 
true that almost all (except a finite number) perfect Hessenberg NRS-matrices of a given 
Hessenberg type are reduced? For which Hessenberg types is the number of such matrices 
finite? Find the numbers in such cases. 

The next two problems are actual in the cases when the number of nonreduced matrices 
is finite and infinite respectively. 

First we show a small table of conjectured numbers of nonreduced perfect matrices for 
given Hessenberg types. We list all the types of Hessenberg complexity less than 5. The 
Hessenberg complexity of Q is denoted by i;{fl), the conjectured number of nonreduced 
NRS-matrices of this type is denoted by #(fi). 



Q 


(0,10,0,1) 


(0,11,0,2) 


(0,11,1,2) 


(0,11,0,3) 


(0,1 1,1,3) 


(0,11,2,3) 


,{n) 


1 


2 


2 


3 


3 


3 


#(fi) 





12 


12 


6 


10 


10 



Q 


(0,12,0,3) 


(0,12,1,3) 


(0,12,2,3) 


(1,2 0,0,1) 


(0,11,0,4) 


(0,11,1,4) 


,(fi) 


3 


3 


3 


4 


4 


4 


#(fi) 


14 


10 


10 


94 


6 


8 



Q 


(0,11,2,4) 


(0,1 1,3,4) 


(0,13,0,4) 


(0,13,1,4) 


(0,13,2,4) 


(0,13,3,4) 


,{n) 


4 


4 


4 


4 


4 


4 


#(fi) 


10 


8 


10 


12 


8 


8 



At this moment it is only known that there are no nonreduced matrices of type ((0, 1|0, 0, 1)). 

Problem 2. Study the asymptotics of the number of reduced operators with respect to 
the growth of the Hessenberg complexity. 
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Consider now the family NRS(^{0, 1|1, 1, 2)) shown in Figure 4. Note that any operator 
with matrix 

^(o,i|u,2> (-2^^ -2t - 1) for t > 

and any operator with matrix 

^(o,f|i,\,2) (2m, 2m^ - 1) for M > 

is a square of some other 5*^(3, Z)-operator. All these operators are contained in two 
parabolas at the "boundary" of the family A^i?S'((0, 1|1, 1, 2)). Theoretically a similar 
situation can take place in the case of nonreduced operators. In the case of negative 
answer to Problem 1 it is interesting to know the answer to the following question. 

Problem 3. Is it true that for any Hessenberg type the set of nonreduced perfect matrices 
is contained in a finite number of parabolas? How many infinite series do we have for a 
given Hessenberg type? 



6.2. Three dimensional RS-matrices. We study the real spectrum case (of RS-matrices) 

(1,0,1) I 
(0,1|1,0,2)' 



in a particular example of the family of Hessenberg matrices HLi!-. f. „Jm, n 




^(0,1|1,0,2>("^'"') - 

By definition, the Hessenberg complexity of the matrices of the family equals 2. Hence 
a Hessenberg matrix H,Q\:-^^Q^Jm,n) is reduced iff it is not conjugate to some matrix of 
the unit Hessenberg complexity, i.e., to some matrix of Hessenberg type (0, 1|0, 0, 1). 

By Proposition 3.4, the matrix HL\L\^^Jm,n) is reduced iff the set of absolute values 

of the MD-characteristic of the operator with matrix HL\L^^Jm,n) does not attain the 
value 1 at integer points. 

In Figure 7 we study the matrices H^^Q\L'^^Jm,n) with 

-20 <m,n< 20. 

The square in the intersection of the ?72-th column and n-th row corresponds to the matrix 
HL '^i'-^ q 2) (^^? ^) • The square is colored in black if the characteristic polynomial has rational 
roots. The square is colored in gray if the matrix is irreducible and there exists an integer 
vector (x, y, z) with the coordinates satisfying 

-1000 <x,y,z< 1000, 

such that the MD-characteristic of the operator with matrix iJ(o,i|i,o,2>(l, 0, l)(m, n) at- 
tains the value 1 at (x, y, z). All the rest of the squares are white. 

If the square is gray, then the corresponding matrix is not reduced. If a square is white, 
then we cannot conclude whether the matrix is reduced or not (since the integer vector 
with MD-characteristic equal to 1 may have coordinates with absolute values greater than 
1000). 
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TO — —n — 1 



x(-i) = o 




TO, = n — 1 



x(i) = o 



Figure 7. The family of matrices of Hessenberg type (0, 1|1, 0, 2). 



From Figure 7 the NRS- domain can be easily visualized. This domain is almost com- 
pletely consists of white squares. We draw a boundary broken line between the NRS- 
squares and the other squares with gray. 

Remark 6.1. We have checked explicitly all the periods for all sails of the continued 
fractions for operators with matrices HL\!-^'q ^.{m, n) with 

-10 <m,n < 10. 

These matrices are contained inside the black square in Figure 7. It turns out that the 
exact result for this set completely coincides with the above approximation. If the square 
is gray, then the corresponding matrix is not reduced. If the square is white, then the 
matrix is reduced. 



Statement 6.2. If an integer m+n is odd, then the matrix HL\LQ^Jm,n) is reduced. 
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Proof. Note that the MD-characteristic at a vector [x, y, z) for the operator with Hessen- 
berg matrix H,Q'-^,^\^^Am.,n) equals 

I - 2x^ - (4n + 2)x'^y - (4n^ + 4m + 'in)x^z + (4m + 2)xy'^+ 

{4mn + 2m + 6n + 6)xyz + (— 2m^ + 4n^ + 2m + 6n + 2)x2;^ — 2y^— 

(2m + 2n + 2)|/^2; - (2mn + 2n + 2)|/2;^ + (m^ - n^ - 2n - 1)2;^ |. 

From this expression it follows that the MD-characteristic with odd m+n takes only even 
values at any integer point {x,y,z). D 

Remark 6.3. Now we can say nothing about the asymptotic behavior of RS-matrices. 
From one hand Statement 6.2 implies that there exist rays of reduced operators. From 
the other hand all matrices corresponding to integer points of the lines 

l)m = n; 2)m = n + 2; 3)m = — n; 4)m = — n — 2; 5)n = 3m — 4; 6)m = 3n + 6 

are reduced (we do note state that the list of such lines is complete). 

So the following problems arise. Denote by Sp the square 

{(m, n)\ — p < m,n < p}. 

For a fixed Hessenberg type Q denote by Qn{p) the ratio of the number of reduced matrices 
of type fl corresponding to integer points of Sp to the number of all integer points of Sp 
(i.e.,to(2p+l)2). 

Problem 4. For any Hessenberg type Q find all limit points for the sequence Qq{p), 
p = 1,2,3,... Give the upper and the lover estimates for the set of limit points of the 
sequence. 

It is more likely that the limit exists and 

lim Qn{p) = 1. 

p— »+oo 

6.3. Four-dimensional case. As we have already studied, the families of NRS-matrices 
in S'L(3,Z) for distinct Hessenberg types has similar structures (see in the proof of Corol- 
lary 5.14). For instance, they are contained in the union of two parabolas. The reason 
for that is the 5*^(3, Q)-congruence of all the families with expanded integer parameters 
to rational parameters. In higherdimensional cases we have the same situation. 

In this section we take a first glance at 5*1/(4, Z), studying a family of Hessenberg 
matrices of Hessenberg type (0, 1|0, 0, 1|1, 3, 1, 4) of Hessenberg complexity 4. Let 

/ 1 n \ 



r(0,l,0,l) 



-"(0,1|0,0,1|1,3,1,4>'^'' '"' "-J 

Further we write H{1, m, n) instead of -f^(o'i|o o^h 3 1 4\(^5 ""^^ '^)) for short. 



1 3 3n + / + l 

Oil n + m 

y 4 4n + 1 J 

r(0,l,0,l) 
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a) Two real roots; 




X I 

b) No real roots; 



„==„. 



,-.< 

'<:. 



c) Nonquartic irrationalities. 
Projection to the plane I = 0. 



Figure 8. The family of matrices of the Hessenberg type (0, 1|0, 0, 1|1, 3, 1, 4). 



In Figure 8 we show the family of matrices H{l,m,n) with integer parameters l,m,n 
satisfying 

—15 < l,m,n < 15. 

An operator H{1, m, n) is represented by the cube with unit edges parallel to the axes and 
with center at point [1,171,11). Gray cubes of Figure 8a) correspond to the matrices with 
two complex and two real eigenvalues. Light-gray cubes of Figure 8b) correspond to the 
matrices with two distinct couples of complex-conjugate eigenvalues. 

As before, we study the set of matrices with irreducible characteristic polynomial over 
rational numbers. So we get the following restrictions on admissible triples {l,in,n). 

First, the characteristic polynomial does not have rational roots. This is equivalent to 
the following: the integers 1 and —1 are not the roots of the polynomial. The characteristic 
polynomial for the operator H{1, m, n) in t is 

t^ + (-4n - 2)t^ + (-4m - 2)t^ + (2 - 4/)t + 1. 

Then, the number 1 (or —1) is a root of the characteristic polynomial iff n+m+l = (or 
n—m—l = respectively). 

Secondly, the characteristic polynomial is not a product of two polynomials of degree 2 
with rational coefficients. There are exactly two series of such decomposable polynomials: 

(t^ + at + l)(t^ + bt + 1); (t^ + at- l)(t^ + bt - 1), 

where a and b are integers. The characteristic polynomial of the matrix A{l,m,n) with 
integer parameters {m,l,n) is decomposable into two polynomials of degree 2 with ratio- 
nal coefficients iff at least one of the following systems has an integer solution in (a, b) 
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iables: 
f (-6 -4/ + 2)6 + 2 = 




} -6-4/ + 2 = 


a 


l-n-1 = 






-4m - 2 



-b + 4l- 2)6 - 2 

-6 + 4/ - 2 

/ + n 



-4m - 2 

a 





All the solutions of the above systems correspond to certain cubes with centers lying on 
the plane l—n—1 = and l+n = respectively. The projection along the /-axis to the 
plane / = of all cubes corresponding to the integer solutions of the first system coincides 
with the projection of the cubes of integer solutions of the second system. In Figure 8c) 
the projections of these cubes are dark-gray squares. 

We do not worry that the discriminant of the characteristic polynomial is nonzero, since 
it is zero only if the characteristic polynomial is reducible over rational numbers. 

We conclude with several open questions. 

Problem 5. Study an asymptotic behaviour of the set of nonreduced operators in the 
families of perfect Hessenberg operators in SL{4, Z) with fixed Hessenberg types. 

The answers to Problem 5 are interesting for the subfamilies of the matrices with 
four real eigenvalues, with two real and two complex eigenvalues, and with four complex 
eigenvalues. 

It is also interesting to know the answers to the following questions: 

Q.l. Is it true that there is a unique asymptotic direction for the family of Hessenberg 
matrices with all four non-real eigenvalues with fixed Hessenberg type? 

Q.2. How many parameters does the family of asymptotic directions for the case of 
Hessenberg matrices with two real and two complex eigenvalues for a fixed Hessenberg 
type have? 

Q.3. Is it true that the set of asymptotic directions of Hessenberg matrices with all real 
eigenvalues is everywhere dense in the space of all asymptotic directions with the natural 
topology? 

Q.4- Is it true that for any Hessenberg type there exist only finitely many nonreduced 
Hessenberg matrices with two real and two complex eigenvalues {with four complex eigenva- 
lues) for a fixed Hessenberg type? 
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